Quantum phase transition in Bose-Fermi mixtures 
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We study a quantum Bose-Fermi mixture near a broad Feshbach resonance at zero temperature. 
Within a quantum field theoretical model a two-step Gaussian approximation allows to capture the 
main features of the quantum phase diagram. We show that a repulsive boson-boson interaction is 
necessary for thermodynamic stability. The quantum phase diagram is mapped in chemical potential 
and density space, and both first and second order quantum phase transitions are found. We discuss 
typical characteristics of the first order transition, such as hysteresis or a droplet formation of the 
condensate which may be searched for experimentally. 
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I. INTRODUCTION 

Experiments with ultracold quantum gases provide 
an attractive new way to study many-body physics of 
neutral particles with short-range interactions. Con- 
siderable progress in understanding the phenomena of 
Bose-Einstein condensation for bosons and the BCS-BEC 
crossover for fermions are among the key successes of the 
field [l|. On the other hand, many-body mixtures of par- 
ticles with different quantum statistics, i.e. Bose-Fermi 
mixtures, are not as well understood theoretically and 
are believed to exhibit very different behavior to pure 
Bose and Fermi systems. Moreover, recent experiments 
allowed to prepare and study mixtures of bosons and 
fermions in the quantum degenerate regime, thus lead- 
ing to direct experimental tests of theoretical predictions 
for these mixtures. 

Early theoretical studies were mainly focused on 
weakly coupled systems, both isotropic and trapped 
[2|, |3| . Bose-induced fermion pairing in strongly-coupled 
Bose-Fermi mixtures was studied in [^l] . Advent of Fes- 
hbach resonances provided an experimental stimulus to 
develop theoretical descriptions of strongly interacting 
Bose-Fermi mixtures. First, properties of an individual 
boson-fermion Cooper pair embedded in the many-body 
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FIG. 1: (Color online) Transition from a non-interacting mix- 
tures of bosons (shaded blue) and fermions (solid red) to 
a strongly interacting system where fermionic molecules are 
formed. For the density balanced case illustrated here, the 
cross marks the quantum critical point QCP where the Bose- 
Einstein condensate vanishes. 



environment were studied [5, o] . Subsequently, a num- 
ber of theoretical studies has been undertaken to address 
both narrow [3-[2| and broad resonances [1Q|-[13J . On the 
experimental side, enhanced three-body recombination 
was used as an efficient tool for the identification of a 
number of Feshbach resonances in Bose-Fermi mixtures 
(for review see [14]). 

In this article we consider a mixture of bosons 
and fermions whose interaction strength can be tuned 
through a Feshbach resonance at zero temperature T = 0. 
The theoretical formalism presented in this work is ap- 
plicable for the description of resonances with arbitrary 
width. But since recent experiments with Bose-Fermi 
mixtures found relatively broad resonances, our main re- 
sults are obtained for Feshbach resonances in the limit of 
infinite width. 

If the attraction between bosons and fermions is the 
only relevant interaction, the general picture of the be- 
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FIG. 2: (Color online) Sketch of the quantum phase diagram 
in the space of n^/n^ vs (a/cp)"^ for a small repulsive boson 
interaction clb — as /a — 0.17. The first order phase tran- 
sition separates the symmetry broken phase (BEC-LIQUID) 
from the symmetric phase (NORMAL). The region between 
the two solid red lines corresponds to a mixed state where 
the two phases coexist. In this regime the second order phase 
transition line (blue dashed) separates the metastable (MS) 
normal and BEC phases. 



havior of this system at zero temperature seems to be 
quite intuitive and is schematically illustrated in Fig. [TJ 
For weak attraction between bosons and fermions one 
expects to find a Fermi sphere for the fermions. The 
bosons will, up to a depletion caused by purely bosonic 
quantum fluctuations, occupy the ground state and form 
a pure Bose-Einstein condensate (BEC). As one increases 
the attraction between the two distinct atoms, a bound 
state consisting of one boson and one fermion can form. 
If the number of fermions is larger than the number of 
bosons, the Bose-Einstein condensate will vanish at some 
point as all bosons will pair with fermions. This point 
marks a second order quantum phase transition. 

Our investigation reveals, however, a competing ef- 
fect, namely an effective attractive interaction between 
bosons which is induced by the fluctuations of fermion- 
boson bound states in the presence of a BEC. If one re- 
stricts the analysis to the regime with a small conden- 
sate, the effect of the attractive fermion-boson interaction 
described above dominates and can lead to a vanishing 
BEC for large enough interaction strength. On the other 
hand, for a large BEC the induced boson-boson interac- 
tion becomes important. It turns out that the quantum 
phase transition introduced in the preceding paragraph 
describes actually only a metastable state. For the den- 
sities and interactions near the phase transition of the 
metastable state a quantum state with a large BEC has 
a much lower grand canonical potential. It turns out 
that in this true ground state which we call the "BEC- 
liquid" the fluctuation-induced boson-boson attraction 
must be balanced by a microscopic repulsion between 
bosons. Thus, no stable ground state without a micro- 
scopic repulsive interaction between bosons exists within 
the vahdity of the model. 

In Fig. [2] we depict the sketch of the zero-temperature 
phase diagram, parametrized by the density ratio of 
fermions and bosons ^ and the dimensionless Bose- 
Fermi interaction strength akp^ which emerges from our 
investigation for a fixed small boson-boson microscopic 
repulsion. In this case the normal and BEC-liquid phases 
are separated by a first order phase transition. At the 
first order phase transition the mixture is in chemical 
equilibrium which corresponds to fixed chemical poten- 
tials. On the other hand, the densities undergo a discon- 
tinuous jump as the transition is approached from the 
different phases by varying the Bose-Fermi interaction 
strength. In Fig. [2] the phase transition is thus depicted 
by two red solid lines, and the entire region between the 
two curves represents a mixed state where the two phases 
coexist. The second order quantum phase transition in- 
troduced in Fig. [T]is illustrated by the dashed blue curve 
in Fig. [2] and separates the metastable normal and BEC 
phases. At higher bosonic repulsion the coexistence re- 
gion will shrink. One may guess that at some critical 
value the two red curves will merge with the second-order 
dashed line inducing a second order phase transition. 

As a consequence of the first order quantum phase 
transition an interesting hysteresis effect could be found 



experimentally without changing the temperature (at 
T c^ 0). In particular, one expects sudden jumps in the 
superfluid density as a function of a continuously varying 
magnetic field (Bose-Fermi interaction a) for fixed num- 
bers of fermionic and bosonic atoms. These jumps might 
appear at different values of the magnetic field depending 
on the previous evolution history of the system. 

To demonstrate this, we may follow what happens if 
we decrease the strength of the boson-fermion attrac- 
tion at fixed densities. This can be realized experimen- 
tally by tuning the magnetic field near a Feshbach res- 
onance. Starting with a large attraction corresponds 
to large (a/cp)"^ in Fig. [2l For n^ > rici) the normal 
phase without a condensate where all bosons are bound 
to fermions is stable. As we cross the phase boundary of 
the first order transition, the new ground state becomes 
the BEC-liquid with a large BEC. At the critical chemical 
potential the pure BEC-liquid state has a substantially 
larger density (for given bose-fermi scattering length a) 
than the normal state. At the transition the state with 
the lowest grand canonical potential switches between 
two points that share the same chemical potential on the 
respective first order transition lines. As an example, we 
have depicted in Fig. [2] two such corresponding points by 
full circles. 

For the fixed densities n^ and n^ an immediate tran- 
sition to the new ground state is impossible. In this case 
a further increase of the parameter a beyond the critical 
value leads to a mixed state (black dotted line in Fig. 
[2]), where a fraction of the atoms is in the BEC-liquid 
state, while the remaining part stays in the normal phase 
[15]. Only once the black dotted line crosses the second 
red line, all atoms will be found in the new ground state, 
which is indicated by the square in Fig. [2l While the sys- 
tem traverses the black dotted line in the mixed phase, 
the state of the atoms in the BEC-liquid moves on the 
transition line from the circle to the square. 

So far, the evolution between two phases seems to be 
fully reversible with no hysteresis possible. However, if 
the boson-fermi interaction strength a is only moderately 
larger than the critical value where the normal phase 
ceases to be the ground state, a large grand canonical 
potential barrier separates the normal and BEC-liquid 
states - similar to the vapor-water transition. This bar- 
rier typically suppresses the transition to the new ground 
state - the atoms are caught in a metastable homoge- 
neous state, analogous to supercooled vapor. By further 
increasing a at given density we may cross the quantum 
phase transition in the metastable phase, where a small 
BEC sets in continuously. This is depicted by a star on 
the blue dashed line in Fig. [2l 

As a increases (moving left from the full circle on the 
black dotted line in Fig. [2j), the potential barrier be- 
tween the metastable state and the BEC-Hquid dimin- 
ishes. In consequence, the probability of a transition 
from the metastable state to a state in the mixed phase 
increases. This transition is typically a rather rapid pro- 
cess meaning that there will be some value of a where 



suddenly a large BEC forms. The jump in the conden- 
sate may yield an interesting experimental signature for 
the first order quantum phase transition. For the partic- 
ular case where the jump sets in exactly at the second 
order quantum phase transition in the metastable phase, 
we indicate the state of the mixed phase by the two empty 
circles on the corresponding first order red lines in Fig. 
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In the other direction, starting from a large a in the 
BEC-liquid phase, we may again encounter a metastable 
state, now as a BEC-liquid. It may be necessary to de- 
crease a beyond the critical value for the first order phase 
transition before the system jumps to the mixed phase. 
We observe that the transition between the two phases 
is path-dependent and thus we expect a typical hystere- 
sis effect. Interestingly, this hysteresis may be observed 
as a function of a varying magnetic field (varying a) at 
fixed temperature (e.g. T = 0). It is in this respect the 
same as a first order phase transition in magnets, with 
the jump in magnetization replaced by the jump in the 
conensate. By continuity, it should also be possible to re- 
alize this hysteresis effect by a variation of temperature 
at fixed a. 

The main subject of the present work is the derivation 
and thorough analysis of the above-described quantum 
phase diagram of the Bose-Fermi mixture near a broad 
Feshbach resonance. The paper is organized as follows: 
In section |II] we present the two-channel model describ- 
ing the quantum Bose-Fermi mixture and introduce our 
formalism for treating this system. In section IIIII a short 
discussion of renormalization and vacuum properties of 
the model can be found. We show how to compute par- 
ticle densities in section |IVl The following sections |Vl 
and [Vl] are devoted to the exploration of the quantum 
phase diagram. We present a detailed discussion of the 
metastable state and the associated second order phase 
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transition in section rVTll Finally we present our conclud- 
ing remarks in section IVIIIi The details of the calcula- 
tion of the inverse composite particle propagator and the 
density distributions can be found in the two appendices. 
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where the coordinate-space integral at vanishing temper- 
ature is given by J^ = J^ dr J d^x. The action ([!]) is 
a field-theoretical realization of a two-channel model of 
a Feshbach resonance with (/) and tp denoting scattering 
atoms in the open channel and ^ representing a molecular 
state of the closed channel. To the field ^ we therefore 
assign the mass m^ = r?i0 + tti^ and the (bare) chemical 
potential /i^ = /i^ + /i^ . The bare detuning u determines 
the interaction strength between elementary bosons and 
fermions and will be related to the boson-fermion scatter- 
ing length a in section Hill In addition, s-wave scattering 
of two elementary bosons (j) is allowed with the coupling 
strength A. Elementary particles (j) and i/j are coupled to 
the composite molecule ^ through the Yukawa term with 
the coupling h. This parameter is related to the width of 
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the Feshbach resonance AB through AB ~ ^ — , where 
A/iM denotes the difference in the magnetic moments of 
the particles in the open and closed channel. 

We mention here that in the broad resonance limit 
/i ^ oo, z/ ^ oo, the molecular inverse bare propagator 
is dominated by the detuning term 
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In this limit the action ([T]) follows directly from a theory 
with only elementary bosons and fermions and a pointlike 
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interaction of the form ~ —ip^ip (/)*(!) through a Hubbard- 
Stratonovich transformation. This one-channel descrip- 
tion of the Bose-Fermi mixture near a broad Feshbach 
resonance was used before in [lOL lll|. 

The microscopic model in Eq. ([T]) has a number of in- 
teresting symmetries. Besides the usual symmetries as- 
sociated with translation and rotation, this includes in 
particular two global U{1) symmetries ?7(1)0 x ?7(1)^ 
acting on the fields according to 



II. MODEL AND METHOD 



In quantum field theory the microscopic model of the 
Bose-Fermi mixture is defined by a classical action which 
is a functional of a bosonic field 0(x) and the fermionic 
(Grassmann) fields iIj{x) and ^(x). In the grand canonical 
ensemble employing the imaginary time formalism the 
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(3) 



The associated conserved charges are the particle num- 
bers of elementary bosons (j) and fermions ip. We note 
here that due to its composite nature, the field ^ does 
not have an independently conserved particle number. 



The analytic continuation of Eq. ([T]) to real time is 
also invariant under Galilean boost transformations as 
well as under an "energy shift" symmetry which basically 
redefines the absolute energy scale. For details we refer 
to discussions of similar models in the literature [16|, |l7| . 

In order to obtain the thermodynamic properties of the 
system in the grand canonical ensemble, we need to com- 
pute the grand canonical potential Qg = —pV^ where p 
denotes the pressure of a homogeneous system of volume 
V. In this work we apply a Gaussian approximation to 
determine the effective potential U{p) with p denoting 
an absolute square of the constant background bosonic 
field. For thermodynamics the effective potential is a 
very useful function because its (local) minima determine 
thermodynamically (meta)stable states. In particular, if 
U{p) has a minimum at p = po, the grand canonical po- 
tential of the corresponding state can be determined from 
Qg = VU{po). In addition, Bose-Einstein condensation 
occurs for po > 0, where po determines the condensate 
density. 

In the following we calculate the effective potential in 
two steps. First, we integrate out the fiuct nations of the 
elementary fields, resulting in an effective theory for the 
composite field ^ 
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For this purpose we expand the bosonic field ^ = + 
4= [01 (x) + i02(^)] around its constant part ^ = y^ and 
integrate over the fluctuating fields (/>!, 02, V^ only. In a 
second step we integrate over ^ 
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Here we introduced V = F/T, which must be understood 
in the limit T ^ 0. In this way, the effective potential 
remains finite as T ^ 0. 

Let us explain the procedure in more detail. Due to 
translational invar iance, it is convenient to work in mo- 
mentum space with the inverse Fourier-transform defined 
as f{x) = / e*^^/(p), where / = (27r)~^ /<^Po I ^V Q^^d 
px = pqt -\- p-x [18]. After expanding the action S to 
second order in the elementary fields 0i, 02, V^ and V^*, 
the functional integral (j4]) is of a Gaussian type and can 
easily be performed analytically. By expanding the result 
to second order in the fields ^ one obtains 
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where the bare inverse boson propagator matrix is 
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FIG. 3: Feynman diagrams [20] representing the last two 
terms in Eq. JS]): (a) A composite particle can supply an 
elementary boson to the condensate such that it becomes an 
elementary fermion. The elementary fermion then absorbs a 
boson from the condensate, which results in the reformation 
of a fermionic dimer. (b) Alternatively, the dimer field ^ may 
split up into an elementary fermion and boson before binding 
once again. 



with a{p) = ^ P(f) -\- Xp and b{p) = a{p) + 2Xp. For 

the bare inverse elementary fermion propagator we use 
Gl {p) = ipo + ^ Piij . Finally, as a result of the func- 
tional integration, the renormalized inverse dimer prop- 
agator in Eq. (|6j) reads 
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The first four terms in Eq.(j8]) correspond to the bare 
inverse propagator of the particle ^ which can be directly 
read off from the action S. The remaining two terms are 
depicted in terms of Feynman diagrams in Figure [3] [20| . 

In the second step we compute the effective action F 
by performing the Gaussian functional integral over the 
composite fermionic field ^. This leads to the well-known 
one-loop formula 
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r [e, p] = 5eff [^, p] + -STr log 5^^^ K, p] . (10) 



The supertrace STr is understood to sum over both mo- 
mentum and internal spinor space, while S^^^ = 

r(-p). 



with (pi{p) = i{p) and (p2{p) 



The effective potential is then obtained from the effective 
action F evaluated at a constant background field. Due to 
the fermionic nature of ^, we find U{p) = F[(f = 0, p]/V, 
resulting in 
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The first two terms correspond to the microscopic po- 

A 



tential which has a global minimum at po = ^ > 



for /i0 > and A > 0. The third term originates from 
bosonic fluctuations and results in a quantum depletion 
of the Bose-Einstein condensate due to purely bosonic 
fluctuations [21]. In the following we neglect this con- 
tribution to the effective potential [22]. The fourth term 
equals to the (negative of the) pressure of the elementary 
free fermions and gives a contribution which is indepen- 
dent of the parameter p. The last term accounts for the 
fluctuations of the renormalized composite field ^. As 
will be demonstrated later, the inclusion of this term is 
crucial for a proper understanding of the quantum Bose- 
Fermi mixture as it is responsible for the appearance of a 
local minimum of U{p) at some po > even for jHcf) < 0. 
We would like to emphasize that in contrast to the 
BCS-BEC crossover for fermions, where mean field treat- 
ment (i.e. neglecting bosonic fluctuations) gives reason- 
able results at T = [23], we believe that the two step- 
procedure described above is necessary for a proper un- 
derstanding of the quantum physics of strongly interact- 
ing Bose- Fermi mixtures. The reason for that is the sim- 
ple observation that the pairing field ^ is a fermion and 
cannot form a Bose-Einstein condensate. Near a broad 
Feshbach resonance the contribution from quantum fluc- 
tuations of the composite field to the effective potential 
U{p) is in fact large, which is why one first needs to 
include the pairing dynamics by calculating the renor- 
malized inverse propagator G7^ . Only subsequently one 
can properly study the influence of pairing fluctuations 
on the Bose-Einstein condensation of elementary bosons 
(/). This is directly achieved by our two-step treatment. 
A similar observation has been made before in 18|. 



III. VACUUM & RENORMALIZATION 

As a consequence of the pointlike interactions in the 
microscopic action S^ the integral ({p) in Eq. (J9j) is lin- 
early divergent. For this reason, the quantum theory 
must be renormalized, which is most conveniently done 
in vacuum, i.e. for vanishing temperature and densities 
T = n^ = 77.0 = 0. Specifically, we regularize the integral 
({p) using a sharp ultraviolet momentum cutoff A. All 
cutoff-dependence can then be absorbed into the bare 
detuning u which is related to a low-energy observable 
- the boson- fermion s-wave scattering length a. In this 
way one can take the limit A ^ oo. In our model defined 
by Eq. ([1]), the scattering in vacuum of a fundamental 
fermion t/; and a boson (j) is described by the tree-level 
bound state exchange process. In particular, one has 
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(12) 



with the reduced mass m^ = m^m(f)/{m^ + m^) of the 
elementary particles and G^{uj,p) the real time propaga- 
tor obtained from analytic continuation of Eq. ([8]) using 
uj = —ipo. The frequency uj must be chosen such that 
the incoming fermion and boson are on-shell. 



The solution of this two-body problem, including the 
choice of the chemical potentials in vacuum, the renor- 
malization of the detuning parameter u and the calcula- 
tion of the binding energies closely resembles the solution 
of a similar problem for two-component fermions. In- 
stead of presenting this in full detail here, we refer to the 
literature (e.g. [2J-[26|) and only state the key results. 

In the regime with /i0,/i^ < and vanishing conden- 
sate po = one finds from Eq. (|8|) an exact analytic 
expression for G7^{p). For large A, it reads 
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The cutoff dependent term is canceled by a corresponding 
counter term in the bare detuning parameter u^ which 
reads 
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relating the parameter u of the microscopic model ([T]) to 
the experimentally accessible scattering length a. 

From Eq. (p!3|) one can obtain the binding energy of the 
dimer state that is formed for positive scattering length 
a > 0. In the broad resonance model this leads to the 
well known result [24] 
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In this work we concentrate on the limit of broad res- 
onances with h ^ oo. The inverse propagator for the 
composite fermions (|8]) is then dominated by the last 
three terms which are all proportional to /i^. In contrast, 
the first three terms ipo-\-^ — M^ can be neglected in this 
limit. Thus, the momentum- and frequency dependence 
of G7^ is completely dominated by quantum fluctuations, 
implying that the dimer particle ^ is an emergent degree 
of freedom. Its origin is the attractive contact interaction 
between elementary fermions ip and bosons (j). 

In a similar fashion, the bare boson-boson coupling A 
can be traded for the experimentally measurable boson- 
boson scattering length as- Specifically, 
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2aBA 
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We refer to the literature for its derivation [2^ . Through- 
out this work we use A = ^^^^ which is the leading order 

m^ ^ 

in aB approximation of the exact relation (p!6]) . 

Note that we now have, apart from the chemical poten- 
tials, determined all parameters of our microscopic model 
in Eq. ([T]). The chemical potentials will be used to fix the 
particle densities in the following section HVl 



IV. PARTICLE DENSITIES 

Since actual experiments with ultracold quantum gases 
are performed at fixed particle number, we discuss in this 
section how particle densities are calculated from the ef- 
fective potential U{p). Our starting point is Eq. (fTTj) to- 
gether with the approximate analytic expressions for the 
composite particle inverse propagator that we display in 
the appendix [A] in Eqs. ()A7| lASp . These expressions 
are valid both in the symmetric phase without a conden- 
sate (po = 0) and in the spontaneously symmetry broken 
phase where po ^ 0. For details of the derivation and the 
limitations of this parametrization, we refer to appendix 

El 

All thermodynamic observables can now be obtained 
from the effective potential (pT]) - the particle density 
equations, for instance, follow by differentiation of U{po) 
with respect to their associated Lagrange multipliers, the 
chemical potentials. For the number density of bosons we 
obtain 
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Note that we need to evaluate all expressions at the equi- 
librium condensate density po that is obtained from the 
global minimum of the effective potential U{p). The first 
term in Eq.([T7j) corresponds to the particle density of 
bosons that occupy the ground state, while the third term 
describes the contribution of bosons contained within the 
composite fermions ^. At zero temperature the second 
term accounts only for the quantum depletion caused by 
the boson-boson-interaction. As discussed in [22], this 
term should be neglected if one consistently applies our 
approximation. 

Analogously, the particle density equation for the 
fermions reads 
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The first term accounts for the fermi sphere of the ele- 
mentary fermions, while the second term again provides 
a contribution from fermionic molecules ^. 

The factor 6"*"^^° appearing in Eqs. (p!7| [T8|) is nec- 
essary for the convergence of the frequency integrations 
and is a direct consequence of the quantization proce- 
dure. When employing the residue theorem, it forces us 
to close the integration contour in the lower po-half-plane. 
By analyzing the expression for G7^{p) in Eqs. ()A7HA8p . 
we find that in principle we need to consider both branch- 
cut and pole contributions: A branch cut contributes as 
long as ^ — fi(f) — fi^-\-2Xpo < 0. In this paper, however. 



we restrict our analysis to the region 2Apo — M^ — MV' > 
(see Appendix [Xj). For this reason, branch cuts never 
contribute in our calculations. In addition to that, the 
integrands in Eqs. ^T7\ [T8|) can have between zero and 
three poles in the lower po-half-plane. We found that one 
needs to consider all three poles to obtain the correct de- 
scription of the system. We determined the positions of 
the poles numerically and used the residue theorem to 
compute the frequency integral. We also observed that 
increasing momentum \p\ results in the poles moving to 
the upper po-half-plane. This cuts off high momenta and 
ensures that the momentum integrations in Eqs. (pn[T8|) 
are ultraviolet convergent. 

At this point we can identify the physical conditions 
which must be fulfilled in the vacuum state. In this case, 
the particle density equations should lead to Ucfy = n^ = 
0. Since the individual terms in Eqs. (pT|) and (p!8|) give 
non- negative contributions, they must vanish separately. 
This implies the conditions po = 0, MV' ^ and /i^+p^ < 
eB for a > in the vacuum state. The last condition is 
a consequence of a vanishing contribution from fermionic 
dimers to Eqs. (dZj) and (ITSl) . 

Finally, we extract the particle density distributions 
and the fermionic quasiparticle dispersion curves directly 
from Eqs. ([171 [l8|) in Appendix [0 



V. QUANTUM PHASE TRANSITION 

In this section we discuss the quantum phase diagram 
of the mixture in the theoretically most simple setting. In 
particular, we concentrate on the density balanced n^ = 
n^ system with equal masses tti^ = m^. 

In this case we can explore the phase diagram as a 
function of two dimensionless parameters, {akp)'^ and 
Q^^ — 9iB_ ^^\\]^ \]^Q Fermi momentum kp defined by 

kp = (67r^n^)^/^. As will be demonstrated later, we 
must consider a positive boson-boson scattering length 
as for stability. In the following we restrict our atten- 
tion to the regime as <C \a\ or equivalently \aB\ <C 1. 

For (a/ci?)~^ -^ — oo the elementary fermions and 
bosons are only weakly interacting. In this regime we 
expect the bosons to occupy the ground state (up to 
a small quantum depletion due to a finite clb) corre- 
sponding to Bose-Einstein condensation. This leads to 
a spontaneous breaking of the global U{l)(f) symmetry 
(/) -^ e*"'^^, tp ^ tp, ^ ^ e*"'^^. For the elementary 
fermions we expect a sharp Fermi sphere such that the 
U{l)^l; symmetry tp — )► e^'^^ip^ ^ — ?► ^, ^ — > e*"^^ remains 
unbroken. For a small but non- vanishing negative param- 
eter akp^ one expects some deviations from this picture. 
In particular, there might be an additional depletion of 
the Bose-Einstein condensate and a smoothening of the 
Fermi sphere by weak Bose- Fermi interactions. Never- 
theless, the symmetry properties of the mixture remain 
unaltered. 

On the other side, for (a/ci?)~^ -^ oo, all elementary 
fermions and bosons are strongly bound into fermionic 



dimer molecules ^. Since in this limit the molecules are 
spinless, pointlike fermions, a local s-wave interaction be- 
tween them is forbidden by the Pauli principle. In our 
approximation where interactions between the compos- 
ite fermions are neglected, they are expected to form a 
Fermi sphere. Hence, there is no Bose-Einstein conden- 
sate of bosons in this limit and both the /7(1)^ and /7(1)0 
symmetries remain unbroken. 

Beyond our approximation there might be p-wave (or 
higher partial wave) induced interactions between the 
composite fermions leading to a more complicated ground 
state at T = 0. For a p-wave superfluid ground state cor- 
responding to a condensate of pairs of fermionic dimers 
^, both the /7(1)0 and the /7(1)^ symmetries are bro- 
ken spontaneously. However, in contrast to Bose-Einstein 
condensation of elementary bosons 0, a discrete Z2 sub- 
group of U{l)c/y remains unbroken. 

In general, we therefore expect a true quantum phase 
transition to separate the regimes at {akp)'^ ^ — 00 and 
{akF)~^ ^ 00 in the density balanced mixture. The or- 
der of the phase transition and the exact critical values 
{akF)c^ [27] depend sensitively on the value of the di- 
mensionless boson-boson scattering length clb- From our 
numerical calculations, we found the phase transition to 
be located at {akF)~^ > for all choices of studied pa- 
rameters. We therefore restrict our discussion to that 
region. 

To identify the order of the phase transition, we cal- 
culate the effective potential U{p) given by Eq. (pT]) . As 
was mentioned in section [III in our treatment U^{p) = 
— J log G7^{p) is the only fluctuation- induced term that 
carries p dependence. This is why the asymptotic be- 
havior of U^{p) as p ^ 00 is of a particular interest for 
the stability of the mixture. We investigated this numer- 
ically and observed that, for a^ = 0, the dimer contri- 
bution U^{p) diverges to negative values according to the 
power-law 



udp) 



for p ^ 00 



(19) 



with the exponent k, ^ 1.6 [28]. In fact, we observed that 
the exponent hz depends weakly on the parameters /i^, 
p^ and a. For the parameters we checked hz G (1.6, 1.7). 
Remarkably z^: > 1, resulting in the effective potential 
U{p) to become unbounded from below for A = 0, i.e. for 
as = 0. This means that for A = the model supports at 
most met ast able states (see section rVlT|) which eventually 
collapse into the state with p ^ 00. In physical terms, 
the ground state prefers to develop a large condensate 
due to induced attractive interactions. Since /^ < 2, the 
effective potential can be stabilized by imposing some 
arbitrarily small but positive value for A. Indeed, this 
changes the microscopic or classical part of the effective 
potential in Eq. (pTj) such that for large p it increases 
according to 



_lim C/(p) = -j5\ 



(20) 



Since the inverse composite propagator G7^ in Eq. ()A7p 
depends on A, we find that the fluctuation induced part 
of the effective potential U^{p) becomes a function of A. 
It was observed, however, that this dependence is mild 
and does not affect much the large p behavior found in 
Eq. (p!9|) . We conclude that a finite positive boson-boson 
scattering length qb plays a vital role in our model, as 
it bounds the potential from below and thus renders the 
system thermodynamically stable. 

The situation may be understood by considering the 
boson-boson scattering in the presence of a condensate. 
The relevant interaction strength is given by the fourth 
derivative of the potential with respect to ^, which con- 
tains a term ^-^. While the microscopic interaction 
is point like and repulsive with strength A, the interac- 
tion induced by fluctuations of the composite fermions 
is attractive for large p, decaying ~ —p'^~^ as p -^ 00. 
For some p the effect of this attractive boson-boson in- 
duced interaction may win over the effect of the attrac- 
tive boson-fermion interaction, which leads to pairing. In 
particular, instead of forming fermion-boson composites, 
which would lower the condensate, the system prefers to 
develop a large condensate with the lower grand canon- 
ical potential Qq- Without the repulsive microscopic 
interaction the mixture would be unstable due to the 
collapse of the attractive bosonic system. Since n < 2^ 
for A > there should be a finite critical value p = po 
for which the minimum of the grand potential Qq is 
reached. We conclude that the behavior of U{p) is gov- 
erned by a competition between the classical contribution 
Uc\{p) = —P(/)P-\- ^p^ and the fluctuation-induced term 
U^{p). Thus, to classify the phase transition to the phase 
with Bose-Einstein condensation in terms of its order, we 
need to study the global properties of the effective poten- 
tial U{p) for arbitrary p>0. 

For (a/c^)"^ -^ 00 and clb > the Bose- Fermi mixture 
is in the normal phase, i.e. with the global minimum of 
U{p) located at po = 0- In general, two scenarios for 
the transition to the phase with a Bose-Einstein conden- 
sate are now possible. One corresponds to a first order 
phase transition where the form of the effective potential 
changes as a function of (a/ci?)~^ such that it first de- 
velops a second (local) minimum at pmin > 0. The point 
{akF)c^ where U{pmin) becomes equal to U{p = 0) marks 
a first order phase transition. Fig. |4] illustrates how this 
scenario is realized in the Bose-Fermi mixture. Strictly 
speaking, the effective potential should be a convex func- 
tion. The expressions we obtained from the Gaussian ap- 
proximation are non-convex (see Fig. EJ. Physically this 
suggests the necessity of a mixed state (phase separation) 
which can be obtained via the Maxwell construction [30| . 
In general, the particle number densities n^ and n^ and 
other thermodynamic observables must be evaluated at 
the global minimum of the effective potential. As the 
global minimum undergoes a discontinuous jump, there 
are discontinuities in the particle densities and (a/c^?)"^ 
across the first order phase transition. 

The other possibility is a second order phase transi- 
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TABLE I: Different phases in Figs. [6] and El 



FIG. 4: (Color online) Effective potential for the Bose-Fermi 
mixture as a function of p illustrating a first order phase tran- 
sition. From top to bottom, the curves correspond to val- 
ues of akp = 0, 2.66, 2.69, while ds = 0.17 is fixed for all 
three curves [29|]. All curves were obtained for equal masses 

7710 = m^p. 
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FIG. 5: (Color online) Effective potential for the Bose- 
Fermi mixture for ds = as a function of p illus- 
trating a second order phase transition. The curves 
from bottom to top correspond to increasing values of 
(akpy^ = 1.43,1.45,1.49,1.55,1.61,1.66,1.67. We normal- 
ized the curves to the fermi momentum A:f,o at po = 0. 



tion. In that case, the minimum of the effective potential 
changes continuously from p = to a positive value as a 
function of {akp)'^ - Also the particle numbers n^ and 
n^ are now continuous functions of {akF)~^- Fig. [5] illus- 
trates how the second order phase transition is developed 
in the met ast able state at a^ = (see sect. I VII I for more 
details). 



VI. QUANTUM PHASE DIAGRAM 

After the detailed analysis of the density balanced 
case in the previous section, we are ready for a discus- 
sion of the full quantum phase diagram of a Bose-Fermi 
mixture with equal masses m^ = m^. In general, the 
phase diagram spans a three-dimensional space and can 
be parametrized by three dimensionless variables. For 
instance, we can scale away the boson- fermion scatter- 
ing length a and use {(l(f)^ flip^CLB), where /i^,^ 






and ds = ^ with cb defined in Eq. ([T5]) . We will 
use this parametrization in this section. Alternatively, 
the phase diagram can be parametrized by the different 

set of dimensionless variables ( ^, (a/ci?)~^, ds ) which 

is more appropriate for a direct comparison with experi- 
ments with ultracold Bose-Fermi mixtures (see section |T] 
for our detailed discussion). 

Although a three-dimensional plot is necessary to map 
the full quantum phase diagram, we resort here to making 
a two-dimensional cut, i.e. we fix ds and plot the phase 
boundary in the chemical potential plane (/i^, fl^). Since 
it would be difficult to present all the details of this cut 
in a single plot, we present two separate figures which 
cover two qualitatively different domains of the chemical 
potential plane. 

In Fig. [6] an exemplary cut at ds = 0.17 is illus- 
trated for the bosonic chemical potential covering the 
range /i^ G (—1.15,-0.85). The black circles mark the 
first order phase transition boundary that separates the 
symmetry broken phase from the symmetric phase (see 
Table I for the definition of the different phases). In 
the spontaneously broken phase one finds n^ > n^ cor- 
responding to the regime BEC2. Note that the phase 
BECi is not visible in Fig. [6l but we found that it is real- 
ized in the Bose-Fermi mixture at more negative bosonic 
chemical potential. The dashed black line is obtained 
from the condition G7^{po = 0,p = 0) = 0. It separates 
the area with non-zero boson density (SYMi and SYM2) 
from the area with n^ = (SYM3 and VAC). In the 
latter case, the fermion density also vanishes for /i^ < 
resulting in a thermodynamic state with no density, i.e. 
the vacuum state (VAC). In the inset of Fig. [6l we 
plot a part of the density balanced (n^ = n^) line (solid 
blue) located in the normal phase. The line terminates 
at (/i0, /i^) = (—1,0), where both n^ and n^ vanish, and 
intersects the phase transition line at /i^ = —0.99 and 
(1^ = 0.035 leading to (akp)'^ ^ 2.5 when approached 
from the normal phase. 

By changing ds we obtained more cuts of the phase 
diagram. Qualitatively, ds > 0.17 leads to an upward 
shift of the phase transition line in Fig. [6l In addition, 
for larger ds our calculation predicts that a part of the 
phase transition line in the window /i^ G (—1.15, —0.85) 
turns to be second order. This is illustrated in Fig. [71 
where ds = 0.21. For this particular choice the order of 
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FIG. 6: (Color online) Quantum phase diagram for as = 0.17 
in the chemical potential plane with /x^ = /i0/|es| and jl^p = 
MV^/k-B I with the different phases defined in Table I. The black 
circles mark the first order phase transition boundary. In the 
inset we illustrate the density balanced blue line n^ = n^p 
inside the normal phase which intersects the phase transition 
red line at (a/cF)""^ ~ 2.5. 
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FIG. 7: (Color online) Quantum phase diagram for clb = 
0.21 in the chemical potential plane with /x^ = /i^/lesl and 
jjiilj — /iv^/|es| with the different phases defined in Table I. 
The black circles mark the phase transition boundary, where 
the red (gray) section is of the first order and the green (light 
gray) section of the second order. 



the phase transition changes exactly at riff) = n^. We ex- 
pect that for a sufficiently large a^, the whole transition 
boundary becomes of second order and can thus be ob- 
tained from the Thouless criterion (see section |VlI]) . On 
the other hand, we found that for clb < 0.17 the tran- 
sition boundary remains of the first order and is shifted 
downwards compared to Fig. [6l At sufficiently small as 
it enters the vacuum phase indicating an instability of 
vacuum with respect to the formation of a condensate. 

We observe that our model predicts that a phase tran- 
sition can happen even for n^ > n^ when approached 
from the normal phase. This is evident from the inset of 
Fig. [6l where a part of the phase transition line bounds 
the region SYM2 with n^ > n^. It remains to be seen 
in future work whether this surprising behavior is a true 
feature of the phase diagram or an artifact of our approx- 
imation 132.1 . 
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FIG. 8: (Color online) Quantum phase diagram for as = 
0.17 in the chemical potential plane with /i^ = /^(^/|es| and 
Av^ — fJ^ip/\^B\- The black circles mark the phase transition 
boundary which changes from the first order (red (gray) line) 
to the second order (green (light gray) line). The different 
phases are defined in Table I. 



A different region of the phase diagram for as = 0.17 
is illustrated in Fig. [8] where /i^ G (—0.2,0.3). In this 
figure the symmetric vacuum phase (VAC) is separated 
from the symmetry broken phase (BEC2 and BEC3) by 
the line of phase transition which changes its order from 
the first (red (gray) line) to the second (green (light gray) 
line) at /i0 = and jl^ ^ —1.6. It is worth noticing that 
we find no normal phase present for /i^ > 0. In fact, 
for sufficiently small fermionic chemical potential, i.e. in 
the region BEC3 in Fig [51 we find a vanishing fermion 
particle density. Since there are no fermions in this re- 
gion, the Bose-Fermi mixture reduces to a pure bosonic 
theory with pointlike repulsive interactions. Our approx- 
imation then is equivalent to the Bogoliubov mean-field 
treatment. The green second order transition line in Fig. 
[8] represents the well-known quantum critical point which 
separates symmetric vacuum from a BEC at /i^ = in 
the pure bosonic theory. 

Since our approximation strategy relies on the small- 
ness of the boson-boson scattering length a^, we ex- 
pect that only the qualitative features of the three- 
dimensional phase diagram are captured correctly by our 
current approach. 



VII. METASTABLE STATE 

As we emphasized in section |Vl the effective potential 
is unbound from below at a^ = 0, and the model ceases 
to be thermodynamically stable. Nevertheless, for a cer- 
tain range of parameters, the effective potential U{p) has 
a local minimum po at or near the origin manifesting the 
presence of a metastable state. In this section we concen- 
trate our attention on this local minimum and a possible 
second order quantum phase transition. We treat the 
state as stable, which is justified provided the decay time 
to the global minimum of U{p) is large compared with the 



10 



timescales of typical experiments. The interesting ques- 
tion of a dynamical tunelling from this state is deferred 
to a future work. 

By working in the symmetric phase where po = and 
/i0 < 0, we can then simultaneously solve the particle 
density equations (pTj) and (fT8|) for the two chemical po- 
tentials at fixed particle densities n^ and n^ as a func- 
tion of the dimensionless quantity {akF)~ • This gives 
the elementary particle chemical potentials /i^ and /i^ 
as a function of the combination {akF)~ (blue curves in 
Figs. [9]and[l0l). 

We can then identify a second order phase transition 
point by the Thouless criterion, which states that the 
bosonic mass term m^ = GT^{p = 0) needs to vanish at 
the critical point, 



f^lprCfP 



m 



-H- 



S^ = 



with the boson self-energy denoted by E^^ 
one finds 



(21) 
For a^ = 



-0-- 



G^{p)G^{p). (22) 



As the bosonic mass term can alternatively be obtained 
from the first derivative of the effective potential with 
respect to the parameter p, Eq. (|2T]) is equivalent to a 
vanishing slope of the effective potential dU{p)/dp = 
at p = 0. We emphasize that the criterion (|2T]) is a local 
condition which can only be applied for a second order 
phase transition. 

By substituting the chemical potentials /i^ and /i^ de- 
termined from solving the particle density equations (pTt 
[T8|) into Eq. ([2T]) . we obtain m^ as a function of {akp)'^ 
(red curve in inset of Fig. [TQ|) . We identify the criti- 
cal point of the quantum phase transition from the zero- 
crossing of this function. It is located at {akp)'^ = 1.659 
for density and mass balanced systems, — = ^^ = 1, 
with vanishing boson-boson interactions a^ = 0. This 
number agrees well with the result recently obtained in 

im. 

To relate our findings to experiments, we also investi- 
gate how a change in the mass- and density ratio and the 
boson-boson scattering length affects the location of the 
critical second order phase transition point {akF)~ - 

Figure [TTI illustrates the effect of the mass ratio ^^ on 

the critical point for a range from ^^ = 0.2 to ^^ = 20 
in the density balanced case n^ = n^ with as = 0. We 
observe that the value of the critical point (akp)' first 
decreases with increasing mass ratio ^^ before approach- 



ing a minimum at a mass ratio of 



5 and gradually 
increasing for large values of ^. In Fig. [12] we show the 



m0 



change of the position of the metastable critical point 
{akF)~ with the density imbalance ^ for m^ = m^ 
and as = 0. Since we expect the critical point to be 
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FIG. 9: (Color online) Fermion chemical potential in the 
metastable normal phase as a function of the combination 
(a/cF)""^ for density and mass balanced systems with boson- 
boson scattering length as = 0- 
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FIG. 10: (Color online) Boson chemical potential in the 
metastable normal phase (blue) as a function of the combi- 
nation (a/cF)~ for density and mass balanced systems with 
boson-boson scattering length as = 0. As the boson mass 
m^ — —jicf) + S^ (inset, red), crosses the horizontal axis, the 
system undergoes a second order phase transition from the 
metastable normal to BEG phase. 



present only for n^ > n^, we restrict our analysis to this 
regime. Our results show that an increasing ratio — 

decreases the value of (akp)' - This is expected intu- 
itively, as an excess of fermions increases the probability 
for a boson to find a binding partner. From the result for 
— ^ 1 we can interpolate to the extremely imbalanced 
case of one boson immersed in a sea of fermions. As the 
quantum statistics for a single particle is immaterial, we 
expect to recover the molecule-to-polaron phase transi- 
tion point which occurs in systems where a fermion of 
one type is immersed in a sea of fermions of a different 
type. We found a value of {akF)~ = 1.21, while the 
established value obtained from the variational calcula- 
tion [33] and non-self-consistent T-matrix [11] is given by 



{akpY^ 



1.27. We note that beyond these approxima- 



tions, a value of {akp)^ 
refined methods |34j] . 



0.9 was obtained with more 
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FIG. 11: (Color online) Critical point as a function of the 
mass ratio of bosons and fermions in the density balanced 
case without boson-boson interactions, as = 0. 
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FIG. 12: (Color online) Critical point as a function of the 
density ratio of fermions and bosons for fixed mass ratio — - — 
1 and vanishing boson-boson interactions as — 0. 
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FIG. 13: (Color online) Critical point as a function of the 
rescaled dimensionless boson-boson scattering length clb = 
aB/oL for mixtures with — ^ = 1 and — - — 1. 
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TABLE II: List of some broad Feshbach resonances (width 
|AB| > IG) realized in experiments. The table lists the mea- 
sured positions of the resonances Bo, their widths /\B and 
the background-scattering length for the bosons in units of 
the Bohr radius, as/ciQ- We predict the location of the crit- 
ical point {akF)~ under the assumption of vanishing boson- 
boson interactions, as = 0, and for density balanced systems 
with n^ — Ucf). Furthermore, we give an estimate (obtained 
from the criterion clb = as/a '^ 0.1) for the density nc below 
which the influence of as on the location of the critical point 
is negligible. 



To investigate the influence of the boson-boson scat- 
tering length aB on the location of the critical point for 
the metastable state, we must consider an additional di- 
agram for the computation of the boson self-energy. The 
self-energy reads 

S^=-i-# U-,. +--..-V-I--. (23) 



Note that the tadpole diagram consisting of a simple bo- 
son loop vanishes at the level of our approximation. Our 
results obtained for n^ = n^ and m^ = m^ are sum- 
marized in Fig. [131 For small values of clb = clb/cLi the 
position of the critical point is almost unaltered by the 
boson-boson interaction. But starting at about cib '^0.1, 



the boson interactions strongly influence the position of 
the critical point. However, we note that this is exactly 
the regime where the assumption of a small boson-boson 
coupHng A used to derive the analytic formulae for the 
inverse composite particle propagator ()A7( lASp might 
become invalid. Nevertheless, we conclude that boson- 
boson interactions have a negligible effect on the prop- 
erties of the metastable state as long as the system is 
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sufficiently dilute, that is n^ < 

Table |TT1 lists some Feshbach resonances realized in ex- 
periments. For these experiments we calculated the posi- 
tion of the associated metastable quantum critical point 
as well as the critical fermion density nc below which 
boson-boson interactions are safely negligible. 

So far, we only investigated the second order phase 
transition approached from the symmetric phase. Below 
the critical point {akF)~ < {akF)~ we resort to a direct 
analysis of the effective potential t/(p), which is plotted 
in Fig. [5] for some fixed values of {akF)~ , where all 
curves approximately correspond to a fixed density ratio 
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FIG. 14: (Color online) Condensate fraction near the 
metastable second order phase transition point as a function 
of (akF)~ for density and mass balances Bose- Fermi mix- 
tures with fixed as = (blue). The red solid curve is a linear 
fit. 
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1 [39] . From that we can determine the location of 

the minimum po of the effective potential U{p) that gives 
the metastable equilibrium of the system. This allows 
for the computation of the condensate fraction pok^ as 
a function of {akp)'^ close to criticality in the sponta- 
neously symmetry broken metastable phase (see Fig. [T4|) . 
The critical point is then obtained from the vanishing of 
the order parameter y^, which yields {akF)~ = 1.659 
in perfect agreement with the result we previously deter- 
mined from the symmetric phase. 

Near a second order phase transition the system is scale 
invariant and is governed by a fixed point of the renormal- 
ization group. It is of great interest to study the behavior 
of our model near criticality and determine the critical 
exponents of the metastable quantum phase transition. 
First, we compute the critical exponent /3* correspond- 
ing to the scaling of the order parameter near the critical 
point. It is defined by 



/po 



(akp) - {akp)'^ 



/3* 



(24) 



From the linear fit in Fig. [TH we read off /3* = |. 

Furthermore, we can infer the critical exponent i^* for 
the scaling of the correlation length 



{akp) — {cLkpY^ 



(25) 



= m 



In particular, since 



the value of the critical exponent 



dU 

dp 



2 _ 
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, we can extract 



from the behavior 

-1 



of the boson mass term m as a function of {akp) m 
the normal phase. From Fig. [10] we find v* = ^. 

Both exponents agree with a standard mean-field the- 
ory. Our result is also in agreement with [7], where the 
effective field theory near the critical point was studied 
in detail for Bose- Fermi mixtures near a narrow Feshbach 
resonance. The authors of [3] found the mean field crit- 
ical behavior with the dynamical non-relativistic critical 
exponent z = 2. 



VIII. CONCLUSION 

In this work we investigated the general structure of 
the quantum phase diagram for homogeneous resonant 
Bose- Fermi mixtures near a broad Feshbach resonance. 
We argued that a naive mean-field theory treatment is 
insufficient and found an adequate description within 
the two-step Gaussian approximation. In principle, this 
method can be straightforwardly adopted for the investi- 
gation of Bose- Fermi mixtures at finite temperature near 
a Feshbach resonance of arbitrary width. 

We found that a repulsive boson-boson interaction de- 
scribed in our model by a positive scattering length a^ is 
essential to ensure thermodynamic stability of the quan- 
tum Bose- Fermi mixture. Direct analysis of the global 
properties of the effective potential allowed us to un- 
cover a rich structure of the three-dimensional quantum 
phase diagram with both first and second order phase 
transitions. Phase separation in the mixed state and the 
hysteresis effect seem to be promising experimental sig- 
natures of the predicted first order phase transition in 
Bose- Fermi mixtures. 

We have not yet discussed in what parameter ranges 
the experimental realization of the first order transition 
from the normal phase to the BEC-liquid is most promis- 
ing. From a theoretical point of view a BEG with a mod- 
erate particle density offers the best chances that possible 
additional physical effects, which go beyond the approx- 
imation of fermions and bosons with pointlike interac- 
tions, play only a minor role. This is a prerequisite for 
the validity of the found stabilization of the BEG-liquid 
by the competition between the fluctuation induced at- 
traction and the microscopic repulsion. 

In addition, we discussed in detail the "thermodynam- 
ics" of a metastable state. We successfully determined 
the location of the second order quantum critical point 
which separates a metastable phase with a Bose-Einstein 
condensate from the metastable normal phase. An in- 
vestigation of the effect of such diverse factors as the 
density and mass ratios and the boson-boson scattering 
length on the location of the critical point provided a 
direct way to relate our findings to current experiments. 
Furthermore, we computed the critical exponents and an- 
alyzed the density distributions of the elementary parti- 
cles. The properties of the quasiparticle excitations both 
in the BEG and normal phase were investigated. 

Let us finally note that we have not yet addressed di- 
rectly the question of local stability of a degenerate Bose- 
Fermi mixture near a broad Feshbach resonance. This, 
however, is of central importance for the experimental 
realization of the quantum phase transitions analyzed in 
this paper. In general, one requires two different condi- 
tions to be fulfilled for stability: 

First, the atom loss rate, which originates from mi- 
croscopic three-body recombination, must be small. In 
general, this can be achieved, if the critical regime is far 
from the Feshbach resonance. From this perspective, the 
most promising systems should have a small mass ratio 
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^^ and a small boson-boson interaction dn- 

Second, the mixture should be stable against mechan- 
ical collapse and thus have a positive-definite compress- 
ibility matrix. The question of mechanical stability of 
a Bose-Fermi mixture near a broad Feshbach resonance 
has been recently studied in [40|. It was found that the 
system becomes mechanically stable for sufficiently large 
positive dimensionless boson-boson scattering length a^ • 
We believe that our discussion of global stability of the 
effective potential is complementary to the local stability 
analysis of [40]. 

As we treated the system perturbatively in a^, our 
results have only a qualitative character for large as- 
Proper quantitative understanding of the quantum phase 
diagram in this regime provides an interesting subject for 
future investigation. 

We conclude that an experimental realization of Bose- 
Fermi mixtures at very low temperatures can offer a large 
variety of interesting phenomena, both for the metastable 
second order phase transition, and the ffist order transi- 
tion to a BEC-liquid. In particular, the mixtures are 
expected to show many characteristic features related 
to the ffist order phase transitions. One may expect 
the mixed phase and in particular droplets of a Bose- 
Einstein condensate that are kept together by surface 
tension even once the trap potential is removed, simi- 
lar to water droplets. Another striking signal could be 
hysteresis effects with the sudden appearance and disap- 
pearance of a condensate with a large number of atoms. 
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Appendix A: Composite particle propagator 

In this appendix we derive an expression for the (in- 
verse) propagator of the composite fermion field ^ based 
on a one-loop approximation that takes fluctuations of 
the fundamental fermion (t/^) and boson field (0) into ac- 
count. In the few-body limit of vanishing particle density 
and temperature, our calculation yields the correct result 
for the binding energy of the fermion dimer as a function 
of the scattering length a > 0. At non-zero density, it ac- 
counts for the contribution of dimers to thermodynamic 
observables such as the pressure and the particle densi- 
ties. 

We start from Eq. (|9|) corresponding to the one-loop 



particle-particle diagram in Fig. [3l By writing 
detGTi(p) 



pI + 


( f 


( . f 


1 —ini 


f' 



/i0 + \p 



2mA 



/i0 + 3Ap 



2mw 



/i0 + 2Xp ^ 

/i0 + 2Ap ) - X^p^ 



(Al) 



and neglecting the last term 
({p) considerably simplifies 



X p^ the expression for 



C{p) = h' / i(po + go) + 



2m^ 



■ ^tlJ 



(-^^0 + 2^ - M0 + 2Xp 



(A2) 



In the following let us ffist restrict our attention to the 
domain 2Xp — /i^ > 0, where the pole due to the boson 
propagator is always in the lower half of the complex qo 
plane. We close the go -integral in the upper half and find 
that the whole expression vanishes unless 



(p + q)' 
2m^jj 



p^ -lmpo> 0. 



(A3) 



After using the residue theorem for the frequency inte- 
gration we are left with the following integral over spatial 
momentum q 



Cip) = e 






■ f^^ 



Impoj 



q ipo 



2m(h 



jp+qy 



■ Pcp - Pij ^ 2Xp 



(A4) 
It is straightforward to compute the remaining momen- 
tum integral /^ = (2tt)~^ j d^q in Eq. ([Al]) for exter- 
nal momentum p = 0. To achieve this goal, we reg- 
ularize the linear ultraviolet divergence by imposing a 
cutoff at the scale \q\ = A. Under the assumption 
Im po — Ptjj — Pcf) -\- 2Xp > 0, we obtain 



C(Po) 



h'^rur 



TT 



A- ^Vxo{po) 




6 {p^l; + Im po 



\/xo(po)arctan 



with xo(po) = 2mr [ipo - Pcf^ - p,p ^ 2Xp]. 

For /i^ + Im Po > the computation of C{Po^P) foi" 

non-zero spatial momentum p is significantly more com- 

licated and was done in the real-time formalism in 

[gI, |10|- At vanishing density we could in principle 
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use analytic continuation of Eq. ()A5p and a Galilean 
invariance argument for this task. However, this is not 
exact at non-zero density. In the following, we never- 
theless derive an approximate expression inspired by the 
Galilei-invariant result at zero density. Specifically, in 
Eq. ()A5p we perform the replacement 



Xo(po) ^X{P) = 2mr 



iPQ 



2m^ 



fi(f) - /J^ip ^ 2Xp 



^ (A6) 

and thus neglect further possible dependence on p. From 
numerical computations of ({p) at p ^ we found that 
this is indeed a reasonable approximation for Impo = 0- 
Finally, using the resulting expression in Eq. (|8]) and 
adapting the parameter v according to the discussion in 
section IIII) we find the following expression for the com- 
posite fermion inverse propagator 



h^ 



rrir 

27Ta^ 


"^"-./^(n) 


P 


27r ^^^^^ G; 


i'iP) 


©M 







(A7) 



5VSa..„(^ 



which is valid in the regime 2Ap — /i^ > and Im po — 
l^ilj - l^c/>^ 2Xp > 0. 

Following the same steps it is straighforward to derive 
the inverse composite propagator 

G7\po,P) 



/l2 






e [/i0 - 2\p] 



mr^2m(i,{ii(i, - 2\p) 



rrir 



\/x{p) arctan 



/2m0(/i0 - 2Xp) 



x{p) 



valid in the domain 2Ap — /i^ < 0, /i^ < and Im po 

p^ - P(f)^ 2Xp > 0. 



n^{p)/n^ 



1.0 L 




FIG. 15: (Color online) Boson density distribution ricf) (p) as 
a function of \p\kp'^ near a metastable second order phase 
transition for density and mass balanced systems at as = 
at (akF)~ — 1.608 (blue (dark gray) line), 1.647 (red (gray) 
line), 1.659 (green (light gray) line). All boson density dis- 
tributions in the symmetric phase (a/c^)"^ > (a/c^)^^ are 
identical to the green (light gray) curve. 



n^{p)/n^ 
ZOh 




FIG. 16: (Color online) Fermion density distributions n^ (p) 
as a function of \p\kp'^ for density and mass balanced 
metastable mixtures at as = 0. In addition to the curves 
shown for the bosons in Fig. [15] at (a/c^)" = 1.608 (blue 
(dark gray) line), 1.647 (red (gray) line), 1.659 (green (light 
gray) line), we also show the density distributions in the 
metastable symmetric phase for {akp)' = 5 (dotted orange) 
and {akp)' = 20 (dashed brown). 



Appendix B: Density distributions 

From the particle density equations we can extract 
the density distributions n^ {p) for bosons and n^ {p) for 
fermions, defined by 



^0 = Po + 






(Bl) 






where the integrands are taken from Eqs. (pTl [T8|) . Our 
results for density and mass balanced metastable mix- 
tures with a^ = 0, presented in Figs. [15] and [161 show 



several interesting features. These features are also visi- 
ble in the dispersion curves of fermion quasiparticles ex- 
tracted from the poles of Eqs. ([T7|) and ([T8|) and plotted 
in Fig. [m 

In the metastable symmetric phase (a/c^)"^ > 
(a/ci?)^^, the boson density distribution assumes the form 
of a Heaviside step function. This is not unexpected, 
as bosons, to our level of approximation, can either 
occupy the condensate or can be bound into effective 
fermionic molecules (cf. Eq. ([T7|) ). As po = in the 
symmetric phase, all bosons need to be absorbed into 
fermionic molecules such that their momentum distribu- 
tion assumes the form expected for an ideal fermi gas 
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1.0 




m-/ 



FIG. 17: (Color online) Dispersion curves of fermion quasipar- 
ticles at {akp)' = 1.652 (thick blue - metastable symmetry 
broken) and (akp)' = 1.659 (red - metastable symmetric) 
for density and mass balanced Bose- Fermi mixtures with van- 
ishing boson-boson interactions as — 0. In the symmetric 
case, two curves are present, one each due to elementary and 
composite fermions. The appearance of the Bose Einstein 
condensate, po > 0, leads to avoided crossing of the disper- 
sion curves, reflecting the mixing of composite and elementary 
fermions due to interactions with the condensate. 



of molecules. The fermion density distributions, on the 
other hand, show two steps. The first step at small mo- 
mentum is due to the Fermi sphere of the elementary 



fermions that give a contribution ~ 9 /i^ — ^^ for 

/i^ > 0. The fermionic composites give rise to another 
step function that ends precisely at the fermi momen- 
tum kp- In the dispersion curves (blue in Fig. [T7|) . 
this feature becomes visible through two zero crossings of 
the dispersion branches of the elementary and composite 
fermions. Moving away from the critical point deeper in 
the metastable symmetric phase, the elementary fermion 



chemical potential /i^ approaches zero (see Fig. [9j). The 
first step then moves to lower and lower momentum un- 
til the fermion density distributions assume the form of 
a single step identical to the boson occupation n(/y{p). 
As expected, in this regime all elementary bosons and 
fermioins are locked up into molecular composites which 
form a free fermi gas. 

In the metastable symmetry broken phase the kink in 
the boson and fermion density distributions (Figs. \TE\ 
[T6|) is due to the mixing of elementary and composite 
fermions and can be understood by considering Fig. |3(a)[ 
Here, a composite fermion supplies its boson to the con- 
densate and becomes an elementary fermion before ab- 
sorbing a condensed boson and becoming a composite 
once again. Alternatively, an elementary fermion may 
take a boson from the condensate and form a fermionic 
composite before returning the boson back to the con- 
densate. This mechanism is also visible from the dis- 
persion curves of fermion quasiparticles (Fig. [T7|), where 
it leads to the avoided crossing of the dispersion lines 
as one moves from the symmetric to the symmetry bro- 
ken metastable phase. This feature was also observed in 

iai. 

We note here that the density distributions we ob- 
tained do not reflect the relative movement of elementary 
particles bound inside fermionic dimers. In this sense 
Eq. (jBip does not correspond to a proper definition of 
occupation numbers. However, it is a rather convenient 
way to analyse and illustrate the expressions for the in- 
tegrated particle densities in Eqs. (pTl [T8|) . This is also 
the reason why we do not encounter a smooth decay of 
the density distributions^ n(p) ^ |p1~^ foi" high values of 
p as predicted by Tan [41]. The authors of [ll] computed 
the proper occupation numbers in momentum space and 
did observe the expected tail. 
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